\section{Introduction}\label{sec:intro}

We consider the problem of how to model the power of a modern multicore processor as a function of the speed of its cores.
On its surface, the problem seems simple.
For instance, let us consider the following natural model, and then compare what it predicts to what happens in an actual experiment.

The classic model of the power consumed by a single core, \PowerSC, as a function of its speed or operating frequency, $\Freq$, is%
  \footnote{For simplicity, \refeq{eq:singlecore} omits the usual constant term to capture static power; the crux of our argument holds with or without it.}
%
\begin{equation} \label{eq:singlecore}
  \PowerSC(\Freq) = \alpha \times \Freq^\beta .
\end{equation}
%
This model appears in a variety of seminal papers on the work scheduling problem, in particular when the system provides dynamic voltage and frequency scaling (DVFS)~\cite{yao1995scheduling,bansal2007speed,albers2014speed}.
%
Given such a model for a single core, it is natural to estimate multicore power, \PowerMC, by adding the power consumed by each of, say, \NumCores cores:
%
\begin{equation} \label{eq:singlecoreextension}
  \PowerMC(\Freq_1, \ldots, \Freq_\NumCores) = \alpha \sum_{c=1}^{\NumCores} \Freq_c^\beta.
\end{equation}
%
Critically, this approach assumes \emph{independence}, in two senses:
an individual core's power depends only on its own speed, independent of what is happening on the other cores;
and the total power is just the sum of power from each core.
This model predicts that accelerating an individual core will incur a proportional power increase.

\begin{figure*}[htbp]
%    \includegraphics[width=0.5\textwidth]{IPDPS_StableWorkload}
\begin{center}
\subfigure[The power curve with frequency]
{       \label{fig:powercurve}
        \includegraphics[width=0.39\textwidth]{maxfredominatepower}
}
\hspace{0.2in}
\subfigure[The power change curve with frequency]
{
        \label{fig:powerdifference}
        \includegraphics[width=0.37\textwidth]{powerdifference}
}
\end{center}
\caption{A motivating example demonstrating that the power consumption of each core is determined by its own core speed \textit{and} the speeds of other
    cores on the same chip.
    %The vertical direction shows the total power consumption over quad-core on the same chip, and horizonal direction shows the time when the given workload is running. The four different color ovals show four nonuniform power phases. %When each core will increase their speed one by one, the increase in power is very different at different step. When each core will reduce their power step by step, the decrease in power is very different at different step either.
    }
    \label{fig:motivation}
\end{figure*}

Now observe how power actually changes in an experiment.
\RefFigure{fig:motivation} shows how the power  of an Intel Core i7-4770K processor (a quad-core Haswell processor) varies over about 250\,second interval under a sequence of frequency changes. In \reffig{fig:powercurve},  all cores are running the same workload repeatedly.
Initially, all cores also run at the same frequency, \GHz{0.8}.
During the first phase (up to just under 50\,seconds), we increase \emph{only} the frequency of core 0 to \GHz{3.5}, resulting in a significant power increase.
During the second phase (up to just under 150\,seconds), we increase the remaining 3 core frequencies, one after another, to \GHz{3.5}, and observe almost \emph{no change} in power. We then run the process in reverse: during phase three (through about 250 seconds), cores 1-3 decrease their frequencies from \GHz{3.5} to \GHz{0.8}, and during phase four (remaining 50 seconds), core 0 also slows from \GHz{3.5} to \GHz{0.8}. \RefFigure{fig:powerdifference} shows the power change with time. We can clearly see that only in phase one and phase four the power is changed. During phase two and phase three, the power change is almost zero. The power varies with frequency \emph{neither} independently \emph{nor} uniformly, as \refeq{eq:singlecoreextension} would otherwise predict.

Thus, the goal of this study is to propose a family of alternative models that more realistically capture how power changes.
All of the models in this family try to capture some notion of aggregate \emph{statistical behavior} of all cores, in contrast to assuming independence and composing single-core models.
%
The models use natural and interpretable parameters, namely, a statistically estimated average speed and some measure of frequency variation, e.g., the difference between maximum speed and average speed.
%
The family's general functional form permits a piecewise structure in these parameters.

We explore this family systematically, to show how one can by experiment ``derive'' a suitable model of multicore processor power.
We carry out these experiments using 28 compute-intensive benchmarks from SPEC2006~\cite{henning2006spec} on five recent Intel multicore processors---based on Nehalem, Sandy Bridge, Ivy Bridge and Haswell microarchitectures and spanning 45\,nm, 32\,nm, and 22\,nm technology nodes---ultimately yielding a power model with an average relative residual of 2.4\% (in absolute value).
These results help bolster the practical case for using our approach.

%<<<<<<< .mine
%From this family, the model that best fits today's multicore processors reflects what appears in \reffig{fig:stableworkload}, namely, that it is really the maximum frequency among the cores that drives overall power consumption.
%Interpreted differently, this suggests that there is effectively just one power domain governing all cores.
%However, if in the future processor architectures evolve so that multiple power domains emerge, the overall family of models we describe can also express that behavior.
%=======
Moreover, from this family, the model that best fits today's multicore processors reflects what appears in \reffig{fig:motivation}.
%\TODO{Check these next two sentences for technical accuracy.}
In particular, we find that a piecewise linear model, with just 1 to 3 segments on the \emph{maximum frequency} among the cores, best expresses overall power consumption.
Interpreted differently, this suggests that there is effectively just one power domain governing the cores, and that linear functions---rather than general polynomial functions, such as squared or cubic polynomials---built over a small number of frequency regions describe the data well.
However, if in the future processor architectures evolve so that multiple power domains or polynomial dependences emerge, the overall family of models we describe can also express that behavior.
In principle, one just reruns our experiments to determine the new model parameters.
%>>>>>>> .r652

We hope these model properties and results will enable future researchers to tackle a variety of scheduling problems in more appropriate analytical frameworks.
Such problems might include classical scheduling problems under DVFS, as well as emerging scheduling problems, such as how to assign work to cores \emph{and} set core speeds to satisfy a power bound~\cite{Patki:2013qf}.
Indeed, we sketch a proof-of-concept example of an algorithm to do the latter in \refsec{sec:tuning-ee}.
